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I • Overview and Summary of Findings t 

The Basic Skills Program held in the Philadelphia schools during the 

Spring and Summer of 1967 consisted of staff development for teachers and 
remediation in reading and arithmetic for first through sixth grade pupils 
underachieving in those subjects. Data were collected bn pupils' achieve- 
ment and attitudes, teachers' attitudes and teachers' professional judg- 
ments of staff developments 

1, For pupils, the main purpose of the program was successfully accom- 
plished, Without exception, all measures of reading and arithmetic achieve 

ment showed significant gain, 

2, No method of instruction emerged as clearly superior to any other 



method, 

3, No measurable attitude changes occurred for either pupils or teachers, 

4, The greatest weakness of the program was staff development. Question- 
naire data collected from teachers suggest two ways in which future staff 

development programs can be improved. 

The first indicated change is initiating recruitment early enough for 
all teachers to receive their appointments in time to take advantage of the 

staff development. 

The second indicated change is in format and content. Teachers stated 
a decided preference for workshops, which allow greater individual partici- 
pation and which provide more opportunities for one to become familiar with 
materials and procedures to be used in the classroom. Research on methods 
for inducing behavioral change show that the type of program most likely to 
be effective is the one teachers themselves advocated, 

5. Lateness in getting started weakened the program in other ways. 
Included in these wao the late arrival of books and supplies at many schools. 
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Hi Introduction 

In June and July, 1967, The School District of Philadelphia had a five- 
week remedial program in basic reading and mathematics skills • Eligible for the 
program were lower and middle school pupils (grades 1 through 6) who were more 
than one year retarded in these skills and who were judged capable of perform- 
ing at their proper grade-levels. Beyond the immediate purpose of raising their 
levels of achievement, the program was designed to Improve their attitudes 
toward learning, and thus to influence their learning in future post-session 
situations. 

Preceding the summer session, during May and June there was a staff develop- 
ment program. Teachers, lead teachers, reading and mathematics specialists and 

/ 

invited authorities on new techniques and materials in education participated. 
This pre-session program was designed to introduce teachers to new methods in 
teaching basic skills, to encourage their use of them and to provide them with 
a better understanding of how to teach basic skills. 

Five reading methods were evaluated. Three of them are experimental. 

These are the Imperial Productions self-directive program (which uses tapes 
and pupil response booklets) ; the Stern structured linguistics (which is com- 
bined with phonics); and the Sullivan programmed reading (which uses a visual 
approach). Schools using either of two non-experlmental methods, traditional 
phonics and eclectic approaches, served as a control group. 

In the middle grades (5 and 6), some schools had Madison Project Mathematics 
Laboratories, These labs present the so-called "new math” by means of discovery 
learning techniques. Schools without labs served as a control group. 
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The innovative materials and techniques evaluated are the following; 
•televised instruction, used in conjunction with the regular 
teaching of' reading. 

•listening centers, electronic devices with attached earphones 
permitting groups of 4 to 8 children to hear lessons, stories 
and directions requiring attention for subsequent performance 
of a task^ 

•Instructional Materials Centers containing books, tapes, 
records, filmstrips, games and other things to be used in the 
instruction of children^ 

•films, filmstrips and tapes used in remedial readings 
•the computing abacus, a device for giving children visual and 
graphic understanding of arithmetic concepts • 

•the 100-chart, a display device for teaching facts about numbers • 
•the Number Line, a device with which concepts relating to the 
number system can be demonstrated • 

• Modern Mathematics Worksheets by Albert Filano 
• Modern Math for- Achievement , worksheets especially designed for 
underachievers in the middle grades, 

•Cuisenaire rods, a set of materials used in the development of 
arithmetic concepts. 

•transparency originals, which can be projected on a screen to 

present new concepts or summarize what has been learned. 

Another Innovation involved personnel. Lead reading and arithmetic 

teachers worked with principals and teachers to assist them in diagnosing 

pupils* difficulties, developing teaching procedures, demonstrating new 
materials and techniques and coordinating school-wide activities. 
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• K.eview of Related Research 

Research on the relative effectiveness of instructional methods 
typically have yielded non-significant results, ilarris (1962, 1964) 
reviewed research on remediax reading programs conducted during the 
school years 196U— 61 and 1962—63 and concluded that substantial gain 
usually results from short-term programs. Uorris (1958) criticized 
research on remediation on the grounds that smaller-than-ordinary 
class size almost guarantees positive results no matter what methods 
are used. Summers and Hubrig (1966) reviewed doctoral dissertations 
xcritten in 1963 and found that 15 of 17 studies relating reading 
methods to achievement had non-significant results. Similar non- 
significance was reported by Anderson, Hughes and Dixon (1956). Siegel 
and Siegel (1967), commenting on the designs of research in education, 
noted that the absence of significant differences between control and 
experimental groups is common. 

The second aspeci: of the program was staff development. Research 
on the acceptance of change indicates that it is facilitated by getting 
people personally involved with the change (bewin, 1958; Coch and 
French, 1948) and by making them aware of its value (Kelly, 1960; 

Rogers, 1962). The first three of these studies involved housewives, 
factory workers and farmers, respectively, but are pertinent in that 
they illustrated principles applicable in all areas of social behavior# 
Kelly provided more immediately usable information; he found that teachers 
who nad been trained in the use of new audiovisual equipment accepted 
it more readily than those who lacked this training. 
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Another line of inquiry relevant to the present study concerned 
teacher effectiveness# Taylor (1961) found that additional training 
in subject matter was the major concern of recent college graduates 
on their first teaching assignment. Mitzel and Medley (1957) found 
that differences in reading improvement among pupils of beginning 
teachers were directly attributable to teacher effectiveness. 

Kerfoot (1967) reviewed 27 investigations of first grade reading 
methods and concluded that teacher effect is far greater than method 
effect and that improvement in reading instruction is more likely to 
result from improvement in teacher effectiveness than from any change 
in methodology. 
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III, Procedures 
A, Samples selected. 

To avoid overtesting any one set of children, reading and arithmetic 
achievement tests were administered to separate samples. Because of the 
excessive amount of time needed to administer both arithmetic tests, the 
arithmetic samples were divided for the posttest administration, each 
subsample receiving only one of the two tests, 

1, The first and third grade reading samples, 

Reading achievement and attitude tests were administered to 
all first and third grade pupils in 20 schools. Selection of these 
schools followed a 2 x 5 design with 2 schools per cell. Social 
class had 2 levels! Lower and Middle, Reading method had 5 levels: 
Eclectic, Traditional Phonics, Imperial, Stern and Sullivan, 

Selection of the specific schools to be used was made by the 
Summer School Committee, Their classifications of schools as Lower 
or Middle Class were verified by 1960 census data for the census 
tra«t in which each school was located. 

The schools from which data were actually collected are these: 



Reading Method 


Lower Class 


Middle Class 


Eclectic 




Adalre 

J.^fferson 


U, Edmunds 
Lawton 


Traditional 


Phonics 


Childs 

McKinley 


Barton 

Fell 


Imperial 




none* 


Holme 

Morton 


Stern 




Drew 

Fulton 


F, Edmonds 
Morrison 


Sullivan 




Kearny 

Reynolds 


Mitchell 

Flnletter 


^Dunbar and Neblnger were originally included, Dunbar participated 
in an experimental study run by Temple University, The Office of 
Research and Evaluation was not informed of this until mid-July, 
at which time it was too late to substitute another school. Material 
fromNeblnger was either not returned or lost in, transit. 






j 

2. The fifth grade reading sample. 

The fifth grade reading sample consists of all classes 

1 

in 8 schools, also stratified by reading method and social class. 



These are: 




• , 


Readlne Method 


Lower Class 


Middle Class 


Eclectic 


. Strawberry 
Mansion 


Bustleton 


Imperial 


Audenried 


FitzPatrick 


Stern 


Gillespie 


Hopkinson 


Sullivan 


Bartlett 


W. Wilson 


The third grade arithmetic sample. 





All classes in 22 schools were administered an attitude test 
and either the Concepts or the Computation subtest of the Stanford 
Diagnostic Arithmetic Test. The 10 schools receiving the Concepts 
subtest are Alcorn, Barry, Creighton, Dick, S. Douglas, Jackson, 

Kelley, McCall, Pennell and Wright. The 12 receiving the Compu- 
tation subtest are Bridesburg, Decatur, F, Douglass, Dunlap, Heston, 

Hanna, Kirkbride, Levering, Mayfair, Morris, Southwark and Whittier. 

As these schools were not stratified on any independent variable, 
analyses were limited to differences between pretest and posttest 
scores . 

4. The fifth grade arithmetic sample. j 

t 

All classes in 10 schools, half of which had Madison Project | 

Laboratories, were also administered an attitude test and either the 
Concepts or the Computation subtest mentioned above. The Concepts 
subsample consists of 3 Lab schools, Furness, Leeds and Shoemaker, 
and one Control school, Tilden. The Computation subsample consists 
of 2 Lab schools, Harding and Masterman, and 4 Control schools, Beeber, 

i 

J 

J 
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Stoddart-Fleisher , Vare Junior High School and Mifflin. 

5. The teacher sample. 

i; 

An attitude test was administered to all first, third and 
fifth grade teachers whose classes were included in any of the 

t • 

I 3 pupil samples. 

6. The teacher questionnaire sample. 

j ‘ 

The post“Session questionnaire was administered to all first,, 
third and fifth grade teachers throughout the city, whether their 
I classes were in a pupil sample or not. Data were collected from 

118 schools. 

5 Note! Wherever classes were ungraded and formed on the basis of pupils' 

j level of mastery of reading and arithmetic, the summer school principal 

selected those classes which contained mainly first, third or fifth graders, 

i 

I as the case may be. 

B. Instruments used. 







i 

1 



! 
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Except for the Post-Session Questionnaire, all instruments were admin- 
istered at the beginning and again toward the end of the five-week session. 

1. The Hotel Phonics Inventory was administered to the first, third and 
fifth grade reading samples. Each pupil's score was the number of correct 
responses he made. 

'J 2. The Reading subtest of the Metropolitan Achievement Test was adrain- 
istered to the third and fifth grade reading samples. Third graders received 

i 

the Upper Primary level of the test and fifth graders, the Elementary level. 

For both grades, alternate forms of the test were used to minimize practice effect. 
About half of each sample received one form for the pretest and the other for the 
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posttest, while the other half received the same forms, but in reversed order. 
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3, The Concepts and Computation subtests of the Stanford Diagnostic 
Arithmetic Test, Level II, were administered to the third and fifth grade 
arithmetic samples, as noted in the preceding section, 

4, The Activities Preference Test, a measure of attitudes toward reading 
and arithmetic, was administered to the first and third grade reading and arith- 
metic samples. The test, which was locally constructed for use in this study, 
was adapted from one constructed by Gurney (1966), Forced-choice items, each 
containing a pair of activities, were used. The pupil's score on the Reading 
Attitude scale consisted of the number of items on which he stated a preference 
for a reading-related activity over a non-scholastic activity. Similarly, his 
Arithmetic Attitude score was the number of items on which he chose an arithmetic- 
related activity. As many children in these samples were preliterate, the two 
alternatives in each item were illustrated. The pupil indicated his responses 

by circling pictures of children performing the activities he himself preferred, 

5, The Middle School Survey, a test also measuring attitudes toward reading 
and arithmetic, was administered to the fifth grade reading and arithmetic 
samples. The Reading Attitude scale, which was originally titled the San Diego 
Inventory of Reading Attitude, was constructed and validated in California 
(San Diego County Department of Education, 1961), The Arithmetic Attitude scale 



was a Likert scale on which the pupil stated how much he liked or disliked 



various arithmetic— related activities, A five— point scale, ranging from Like 
Very Much to Dislike Very Much, was used. Buffer items were included in the list 
of activities, but were not scored, 

'' 6, The Summer Program Adjective Checklist, a semantic differential, was 

administered to all teachers who had classes in the 5 pupil sample. The semantic 
differential is a technique for measuring differences in the connotative meanings 
of words and was devised by Osgood and his colleagues (1957), Teachers rated 
7 concepts, 4 of them general (Teachers, School Children, Myself and Summertime) 
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and 3 of them particular to the Basic Skills Program (Teachers in This Program, 
School Children in This Program and Methods Used in This Program), on the 
major factors found by Osgood. These factors are Evaluation (a good-bad 
dimension). Potency (a strong-weak dimension) and Activity (an active passive 

dimension) • 

7. All first, third and fifth grade teachers were administered the Post- 
Session questionnaire. This instrument used a multiple-choice, sentence- 
completion format. ("I am (a) male; (b) female.") Data were collected from 
118 of the approximately 140 schools open during the summer of 1967. Topics 
covered in the questionnaire included personal and professional characteristics 
of the respondents, experiences they had in the 1967 Basic Skills Program, their 
opinions about staff development and instructional methods and materials, and 
their observations and opinions about their pupils. Teachers were allowed to 
supplement their responses to the items in the questionnaire with free comments 
and explanations. The latter source of information proved as valuable as the 

precoded items themselves • 
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IV. Findings 

A. Achievement tests for pupils. 

In all 5 samples, all posttest means on the achievement measures used 
were significantly higher than the pretest means. This seems to be conclusive 
evidence that children did benefit from the 5 weeks of instruction they received 

These data are presented in Tables 1 and 2 , 



Table 1. 

Achievement Scores for the Reading Samples 



Grade 


Instrument 


N 


Pretest 

Mean 


Posttest 

Mean 


t 


P 


1 


Phonics Inventory 


237 


8.11 


10.34 


3.27 


.001 


3 

3 


Phonics Inventory 
Metropolitan 


375 

457 


37.15 

28.13 


41.84 

29.24 


4.60 

2.31 


.001 

.02 


5 

5 


Phonics Inventory 
Metropolitan 


689 

586 


47.72 

24.88 


51.67 

26.97 


5.98 

3.67 


.001 

.001 






Table 


2. 










Achievement Scores for 


the Arithmetic Samples 






Grade 


Subtest* 


N 


Pretest 

Mean 


Posttest 

Mean 


t 


P 



3 

3 


Computation 

Concepts 


308 

364 


21.40 

35.40 


24.12 

40.64 


7.54 

11.67 


.001 

.001 


5 

5 


Computation 

Concepts 


446 

330 


48.38 

45.53 


51.22 

52.43 


5.74 

13.26 


.001 

.001 


* of 


the Stanford Diagnostic 


Arithmetic Test. 









All 3 reading samples were stratified by social class. On all tests 
administered, analyses of variance showed that middle class children had 
significantly higher pretest means than did lower class children. Further- 
more, on all tests but one (Metropolitan Achievement Test for fifth graders), 
analyses of covariance showed that middle class children had significantly 
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higher gain-scores. Like many variables included in the present study, 
class differences tell us little about the Basic Skills Program, but are 
valuable as a check on the validity of measures used. Pretest, post test 
and adjusted reading achievement test means are in Table 3. 
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Table 3 

Achievement Scores for the Reading Samples by Social Class 



Grade 


Social 

Class 


Instrument 


Pretest 

Means 


Posttest 

Means 


Adjusted 

Means 


1 


Lower 


Phonics Inventory 


7.27 


9.17 


11.25 


1 


Middle 


Phonics Inventory 


11.87 


14.86 


12.78 


3 


Lower 


Phonics Inventory 


30.88 


34.22 


38.03 


3 


Middle 


Phonics Inventory 


40.81 


46.28 


42.48 


3 


Lower 


Metropolitan 


24.50 


24.81 


27.63 


3 


Middle 


Metropolitan 


31.65 


33.58 


30.78 


5 


Lower 


Phonics Inventory 


42.83 


46.01 


49.23 


5 


Middle 


Phonics Inventory 


53.11 


57.80 


54.58 


5 


Lower 


Metropolitan 


21.40 


24.79 


26.86 


5 


Middle 


Metropolitan 


27.46 


28.06 


25 • 99 




The same 


reading samples were 


also stratified by method of instruction 



For the first and third grade samples, no differences attributable to method 
of instruction were found on any achievement test. Analyses of covariances 
of tests taken by fifth graders showed significantly greater gain on the 
Phonics Inventory for pupils taught by the Eclectic and Stern methods and 
on the Metropolitan Achievement Test for pupils taught by the Stern method. 
Pretest, posttest and adjusted means are in Table 4* 
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Table 4« 

Achievement Scores for the Fifth Grade Reading Sample 



I 

I 





by Method 


of Instruction 




Method 


Instrument 


Pretest 

Means 


Posttest 

Means 


Adjusted 

Means 


Eclectic 

Imperial 

Stern 

Sullivan 


Phonics Inventory 
Phonics Inventory 
Phonics Inventory 
Phonics Inventory 


50.21 

48.78 

44.07 

48.81 


55.96 

49.96 
50.55 
51.15 


54.55 

49.45 

52.99 

50.62 


Eclectic 

Imperial 

Stern 

Sullivan 


Metropolitan 

Metropolitan 

Metropolitan 

Metropolitan 


25.49 

22.87 

24.46 

23.91 


23.81 

23o91 

31.83 

26.06 


23.16 

25.01 

31.12 

26.41 



Determination of interaction between social class for the first 



and third grade samples was not possible due to data missing from the 
Dunbar and Nebinger schools. For the fifth grade sample, interactions 

between these two variables on both achievement measures were not 

I 

t statistically significant. 

Analyses of covariance was used to compare Laboratory and Non- 
Laboratory schools in the fifth grade arithmetic sample. On the Computation 
subtest of the Stanford Diagnostic Arithmetic Test, the Lab schools had 
significantly lower gain. No significant difference was found on the 
Concepts subtest. Means on the former measure are in Table 5 . 



Table 5. 

Scores on the Computation Subtest, S.D.A.T., for Fifth Graders 



Treatment 



Pretest 

Mean 



Posttest 

Mean 



Adjusted 

Mean 



Laboratory 

Non-Laboratory 



49.70 

47.35 



50.37 

51.88 



49.51 

52.75 
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B. Attitude tests for pupils. 

When the Activities Preference Test was administered nothing was known 
about its reliability or validity, other than its having face validity and 
appearing to measure what it is supposed to. After pretest and posttest data 
had been collected, a 2 x 2 analysis of variance was done on scores made by 
200 randomly selected pupils. Sex and grade both had 2 levels and there were 
50 pupils in each cell. On the Reading Attitude scale, third graders scored 
significantly higher than first graders, and there was a non-significant trend 
for girls to score higher than boys. On the Arithmetic Attitude scale, boys 
scored significantly higher than girls, and there was a non-significant trend 
for third graders to score higher than first graders, 

Tyler (1956), who reviewed studies on age and sex differences, reported 



findings similar to these. 


Girls, for example, had greater interest in 




reading and boys, in mathematical and mechanical j 


activities. Even though the 


Activities Preference Test still must be regarded 


as experimental 


, we have 


greater confidence in its validity. Data are 


presented in Tables 


6, 7 


and 8. 




Table 6, 










Mean Scores 


on the Activities 


Preference Test 








Reading 




Arithmetic 


N 




First Grade Boys 


8,42 




11,04 


50 




" " Girls 9.20 




9.26 


50 




Third Grade Boys 


9.42 




11,48 


50 




" " Girls 10,02 




9.76 


50 






Table 7. 










Analysis of 


Variance on Reading Attitude Scores 






Source of Variation 


Sum of Squares 


df 


Mean Squares 


F 


P 


Sex 


23.85 


1 


23.85 


2.73 


NS 


Grade 


41.45 


1 


41.45 


4.74 


.05 


Sex by grade 


0o40 


1 


0.40 


0.05 


NS 


Within groups 


1713.40 


196 


8.74 






Total 


1779.10 


199 
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Table 8. 

Analysis of Variance on Arithmetic Attitude Scores 



!. 



Source of Variation 



i- 

r 



Sex 
Grade 
Sex by 



Within 

Total 



grade 

groups 



Sum of Squares 



153.13 

11.05 

0.04 

1832.00 

1996.22 



df 



1 

1 

1 

196 

199 



Mean Squares F p 



153.13 16.63 .001 

11.05 1.19 NS 

0.04 0 

9.21 



i 







1 
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On measures of attitude toward reading administered to A of the 5 samples 
and on measures of attitude toward arithmetic administered to all 5 samples, 
pretest and posttest means were almost identical. The one exception was the fifth 
grade arithmetic' sample in which there was a significant drop on the Reading Atti- 
tude scale of the Middle School Survey. When a large number of measures are used, 
significant differences can occur by chance. This seems to be the case here. 

Thus, the evidence seems conclusive that no changes of attitude took place. These 

data are presented in tables 9 and 10. 

Table 9. 



Reading Attitude Scores* 



Sample 



N 



Pretest 

Mean 



Posttest 
Mean 



First Grade Reading 
Third Grade Reading 
Third Grade Arithmetic 
Fifth Grade Reading 
Fifth Grade Arithmetic 

* Activities Preference Test for Grades 



328 


8.23 


8.11 


0.48 


4.52 


9.17 


9.37 


0.56 


658 


9.86 


9.29 


0.52 


554 


15.04 


14.54 


0.55 


625 


15.35 


14.70 


3.82 


' Grades 


1 and 3; 


Middle School Survey 



NS 

NS 

NS 

NS 

.001 



Table 10. 
Arithmetic Attitude Scores* 




First Grade Reading 
Third Grade Reading 
Third Grade Arithmetic 
Fifth Grade Reading 
Fifth Grade Arithmetic 

* Activities Preference Test for Grades 



332 


10.04 


9.91 


0.16 


NS 


441 


9.75 


10.12 


0.54 


NS 


668 


10.63 


10.47 


1.93 


NS 


542 


43.01 


42.35 


0.73 


NS 


614 


44.54 


43.46 


1.75 


NS 


‘ Grades 


1 and 3; 


Middle School 


Survey 


for 






Analyses of covariance across methods of instruction and social classes 
also showed almost completely negative results* 



C. An attitude test for teachers. 

The semantic differential, which was filled in by 184 teachers, yielded 
21 measures: 7 concepts on 3 scales. For 19 of these 21 measures, pretest and 
posttest means were not significantly different from one another. Teachers as 
well as pupils apparently experience no changes of attitude. The data are 
in Table 11. 

Table 11. 

Summer Program Adjective Checklist (a semantic differential) 



Concept 


Scale 


Pretest 

Mean 


Posttest 

Mean 


t 


P 


School children 
School children 
School children 


Evaluation 

Potency 

Activity 


18.55 

15.53 

18.10 


18.48 

15.90 

18.20 


0.70 

0.19 

0.64 


NS 

NS 

NS 


Teachers 

Teachers 

Teachers 


Evaluation 

Potency 

Activity 


19.52 

17.76 

18.28 


19.57 

17.76 

18.29 


0.91 

0 

0.07 


NS 

NS 

NS 


Summertime 

Summertime 

Summertime 


Evaluation 

Potency 

Activity 


14.83 
16.08 

16.83 


17.08 

16.03 

16.11 


3.21 

0.41 

0.21 


.01 

NS 

NS 


School children in this program 
School children in this program 
School children in this program 


Evaluation 

Potency 

Activity 


17.90 

14.74 

16.63 


18.11 

15.84 

17.40 


0.22 

1.86 

0.67 


NS 

NS 

NS 


Methods used in this program 
Methods used in this program 
Methods used in this program 


Evaluation 

Potency 

Activity 


20.10 

16.69 

14.98 


19.43 

16.69 

17.32 


0.89 

0 

2.60 


NS 

NS 

.02 


Teachers in this program 
Teachers in this program 
Teachers in this program 


Evaluation 

Potency 

Activity 


19.80 

17.49 

17.66 


19.51 

17.39 

18.71 


0.90 

0.54 

1.53 


NS 

NS 

NS 


Myself 

Myself 

Myself 


Evaluation 

Potency 

Activity 


20.27 

16.58 

18.54 


20.27 

17.22 

18.48 


0 

0.44 

0.70 


NS 

NS 

NS 
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To determine whether concepts were rated differently on each of the 
3 scales • one-way analyses of variance were done on pretest scores. Significant 
differences were found on the Evaluation scale (F • 5 .649 P^.Ol) and the 
Potency scale (F » 7.51, p<C.01), but not on the Activity scale. 

The significant F-ratios tell us that one or more concept's were rated 
significantly differently from the others, but they do not indicate which these 
concepts are. For that purpose Newman-Keuls tests were performed on the Evalua- 
tion and Potency scale data. On the Evaluation scale one concept, Sununertjme, 
was rated significantly lower than any other concept} no significant difference 
was found among the 6 other concepts. In absolute terms, Summer time received a 
negative rating and all other concepts, positive ratings. 

Jakobovits (1968), who has been involved with several cross-cultural studies 
in psycholinguistics, found that Summer is universally evaluated highly. A clue 
to the low evaluation given this concept by teachers is the significantly higher 
evaluation they gave it on the posttest rating. The concept’s referent had 
changed in the interim. On the pretest, it meant about a month of work and on 
the posttest, about a month of vacation. Support for this interpretation was 
provided by the free comments teachers made in response to the questionnaire. 

Many stated that the summer session was more enjoyable than they had anticipated, 
which implies that back in June they were not looking forward to it with great 



pleasure. 

On the Potency scale, 4 distinct groupings were found: School C hildren in 
This Program was the weakest concept and School Children (in general), the second 
weakest; Methods Used in This Program . Summertime and Myself were second strongest, 
while Teachers in This Program and Teachers (in general) were strongest of all. 

In absolute terms. School Children in This Program received a negative rating; 
School Children (in general), a neutral rating; and all other concepts, positive 






1 



These latter findings are useful as a check on the validity of the semantic 
differential. We have greater confidence in interpreting negative results as 
evidence that no attitude changes occurred rather than evidence that the instru- 
ment was insensitive to changes which actually may have taken place. 

D. The questionnaire for teachers r 

Data from the Post-Session Questionnaire are presented in 4 ways: 

Responses of the entire sample of 601 teachers are presented first. 

Teachers not stating opinions or not providing information were excluded; in 
every case the percentages add up to 100%. To increase clarity, some items were 
paraphrased and some categories, combined. The actual percentages are in an 

appendix. 

On selected items, chi-square analyses were made on fifth grade teachers 
in school with and without Madison Project Mathematics laboratories. 

On selected items, chi-square analyses were made on differences between 
teachers in the 54 schools using an experimental reading method (Imperial, Stern 
or Sullivan) and those in the 67 schools using a control method (Eclectic or 
Traditional Phonics). Teachers in 17 other schools were excluded from these 

analyses . 

On selected items, chi-square analyses were made on differences between 
teachers with less than 4 years* classroom experience (new teachers) and those 

with 4 or more years * experience (old teachers) . 

Although teachers are overwhelmingly in favor of staff development programs, 
almost half of them did not understand (or were not in sympathy with) the objectives 
of the program and two-thirds of them felt that the program was poorly planned. 

This response is partly attributable to recruitment difficulties. Almost half the 
teachers received their appointments too late for participation in the staff 
development program. Another contributing factor was the format of the program. 
Handwritten comments (supplementary to the questionnaire) indicated that teachers 
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would have much preferred workshops (rather than lectures) in which focus would 

have been on materials and procedures actually to be used in the classroom and j 

J 

in which greater individual involvement would have been possible. 

Over 60% of the teachers claimed that background information on their pupils 
was useful to them. Responses, however, do not tell us how this information 

was used. 

The majority of teachers followed the reading method assigned to them wxth 
some modification, felt that their preparation for using that method was excellent 
or good and would like to continue using the method in the future. 

All but one of the innovative materials and techniques were favorably received 
by teachers. The one exception was televised instruction. Handwritten comments 
indicated that teachers consider television an excellent medium for instruction, 
but thought that the particular programs used last summer were poor. Television 
was rated excellent or good by 40% of the teachers, while other innovations 

received this rating from 69% to 86% of the teachers. 

The item asking teachers whether they would modify their teaching of arith- 
metic was, unfortunately, worded ambiguously and can not be interpreted. 

Teachers perceived their pupils as having improved in the mastery of reading 
and of arithmetic during the summer, which in fact they did. Comparing their 
summer session classes with classes they taught previously, teachers described 
the former as more interested in reading and in arithmetic, as less of a 
discipline problem and as equally receptive to learning. Handwritten comments j 

indicated that teachers attributed this to smaller class size and the opportunity 
for more individual attention. 

There were no significant differences between fifth grade teachers in 
Madison Project schools and in other schools on their perceptions of pupils' 
initial arithmetic level, end-of-session arithmetic level or interest in arith- 
metic. These data are not presented in table form. 



r 

[Teachers using experimental reading methods showed considerable resistance 
to change* They were less likely to feel prepared for using that method^ to 
desire to continue using it and to perceive achievement and interest in their 
pupils* These data are in Table 12* 



Table 12* 

Teacher’s Perceptions of Reading Methods and of Pupils* 



Variable 


Experimental 


Control 


V 


df 


P 


Preparation for 












Method 












Excellent-good 


97 


202 


50*00 


1 


*001 


Fair-poor 


138 


78 








Desire to Continue 












Use 












Yes 


138 


221 


18*80 


1 


*001 


No 


87 


59 








Initial Reading 












Level 












At grade level 


15 


25 


0*81 


1 


NS 


Below grade level 


209 


257 








Final Reading Level 












At grade level 


57 


151 


39*81 


1 


*001 


Below grade level 


163 


129 








Interest in Reading 












More than average 


101 


191 


36*95 


1 


*001 


Same or less 


134 


83 








*N*s for the 


various analyses are 


unequal due 


to missing data* 
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New teachers (those with less than A years* experience) were less likely 
to have used an experimental reading method* Despite this, they also were 
less acceptant of change and were less likely to feel adequately prepared and 
less likely to perceive improvement in reading among their pupils* They did, 
however, describe their pupils as more interested in reading* These data are 

in Table 13* 

S . 

? 

% 

J 



Table 13* 



Relations between 


Classroom Experience 


and other Variables* 




Variable 


New 


Old 


^2 df 


P 




Teachers 


Teachers 




Reading Method 










Experimental 


69 


166 


8*09 1 


*01 


Control 


116 


164 






Preparation for Method 










Excellent-good 


78 


221 


29*96 1 


*001 


Fair-poor 


107 


109 






Desire to Continue Use 










Yes 


119 


240 


3*37 1 


NS 


No 


61 


85 






Initial Reading Level 










At grade level 


9 


31 


3*60 1 


NS 


Below grade level 


175 


291 


- 




Final Reading Level 










At grade level 


43 


165 


38*04 1 


*001 


Below grade level 


139 


153 






Interest in Reading 










More than average 


120 


172 


6*12 1 


*02 


Same or less 


66 


151 






*N*s for the various analyses 


are unequal due to missing data* 
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Although an item about the lead reading and arithmetic teachers was 
inadvertently omitted from the questionnaire, many respondents volunteered 
comments. The praise of the lead teachers was extensive and almost unanimous. 

They were highly regarded as individuals and the services they performed were 
described in terms ranging from "helpful" to "invaluable." 

The majority of free comments took the forms of complaints and recommenda- 
tions. One series of complaints had lateness as its theme. Staff development 
was perceived as a hastily put-together effort. Many teachers were notified 
of their appointments only a few days before the opening day of the session 
and were deprived of an opportunity to participate in staff development. Some 
teachers did not receive books and materials until the middle of the session, 
even though these were needed on the opening day. Recommendations for pre- 
venting recurrences of these problems all stressed getting started earlier. 

Two complaints involved research. Many teachers complained that too 
much class time was taken up by testing. The staff of the Office of Research 
and Evaluation had not been fully aware of this problem. Testing in future 
curricula evaluations should be kept to the absolute minimum consistent with 
good research design. 

A few teachers noted that a definitive evaluation of the various reading 
and arithmetic curricula should not be based on findings obtained from samples 
of underachievers. This is true. We still do not know how normally achieving 
children would perform when taught with these curricula during the regular 

school year. 

Some children made almost perfect scores on pretest achievement tests. They 
were underachieving in either reading or arithmetic, but not in both. Teachers 
suggested excusing them from the subject in which they do not need special help 
and having electives such as art and music for them. 







VI. Discussion 

^he basic purpose of the summer program was to raise the reading 
and arithmetic level of underachieving youngsters^ In this regard, the 
program was an unqualified success. Without exception, every measure of 
achievement showed significant gain. In absolute terms, the gains were 
small, but they seem to have practical significance nevertheless. The 
amount of progress made by pupils in five weeks is encouraging® 



(All methods of instruction were found to be equally effective. One 
should not interpret this as meaning that all methods are equally good 
universally .J Had the type of pupils included in the sample or the period 
of time during which pupils received instruction been changed, some methods 
might have conceivably been found to be more effective than others. 

[interestingly, no measure of attitude showed significant change for 
either pupils or teachers.'' In this instance, learning was not dependent 
Upon attitude change. 

[Teachers using experimental methods were resistant to change^ The 
key to this problem may be the inadequate preparation reported by teachers 



who were using such a method and/or had less classroom experience. All 
social institutions must change to keep pace with cultural and technological 
developments. The school is no exception. Wherever change is initiated, 
the practical problem arises of how to get personnel to accept it. For= 
tunately, the teachers themselves requested the type of staff development 
program which has been empirically demonstrated to do this job best. The 
workshops which they want allow them to become more personally involved, 
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and to get direct feedback on how new methods and materials can be 
adapted to actual classroom situations they will face. 

* Probably, the most important and most practical result of this 

study is the realization that teachers need to be more fully involved 

i in curriculum development and changes. Sarason (1967) found that the 

I 

innovative intent of changes in education is often destroyed by the 
way in which teachers implement them, lie attributed this to the self- 
{ defeating tendency for administrators to introduce changes by fiat and 

I 

i 

to fail to take into account the feelings and opinions of teachers. 
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VI, Appendix 



Responses to the Post-Session Questionnaire (all percentages equal 100/i) 
1, Sex 



Male 26% 

Female 74 

2, Age 

29 or younger 42% 

30 to 39 26 

40 or older 32 



k 



# 



3. Years of Classroom Experience 



Less than 2 years 14% 

2 to 4 years 21 

More than 4 years 65 

4* Previous Summer School Experience? 

Yes 50% 

No 50 

5. Classroom Organization 

Graded 39% 

Ungraded 61 



6. Appointment received early enough for participation in 
staff development? 



Yes 55% 

No 45 



7, Was the preplanning for the program perceived as adequate? 

Yes 32% 

No 68 



8. Do you recommend future staff development programs? 

Yes 80% 

No 20 

9, Were the objectives of the Basic Skills Program clearly stated 
and realistic? 



Yes 57% 

No ^3 
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10. Did you find the background information on pupils in your class to 
be useful? 



Yes 

No 



62% 

38 



11 . 



How carefully did you follow the reading method assigned to you? 



With almost no modifications 
With some modifications 
Not at all 



23% 

70 

7 



12 . 



How would you describe the preparation you received for using 
that method? 



Excellent or good 
Fair or poor 



55% 

45 



13. 



Would you like to use that method during the regular school year? 



As is or with few changes 
With many changes 
Not at all 



71% 

15 

14 



How would you rate each of the following? 



Excellent 
or Good 



Fair 
or Poor 



14. Television 


AOX 


60% 


15. Tapes 

16 . Films 


71 


29 


72 


28 


17. Filmstrips 


78 


22 

18 


18. Listening centers 


82 


19, Instructional Materials Centers 


81 


19 


20. Interdisciplinary ^ pproaches 


71 


29 


21. Language laboratories 


76 


24 


22. Computing abacuses 


83 


17 


23. The 100-chart 


82 


18 


24. The Number Line 


86 


14 


25. Modern Mathematics Worksheets 

(by Filano) 


80 


20 


26. Modern Mathematics for 

Achievement Worksheets 


li\ 


26 


27. Cuisenalre rods 


69 


31 


% 

28. Transparency originals 


72 


28 
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29* At the beginning of the session^ what was the reading level of the 
average pupil In your class? 

At grade level 7% 

Below grade level 93 




i 
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30* At the end of the summer, what was the reading level of the average 
pupil in your class? 

At grade level 27% 

Below grade level 73 

31. Compared with the Interest pupils ordinarily have In reading, 
how would you describe those in your class? 

More Interest 65% 

Same or less Interest 35 

32. At the beginning of the session, what was the arithmetic level 
of the average pupil in your class? 

At grade level 7% 

Below grade level 93 

33* At the end of the summer, what was the arithmetic level of the 
average pupil in your class? 

At grade level 32% 

Below grade level 68 

34. Compared with the interest pupils ordinarily have in arithmetic, 
how would you describe those in your class? 

More interest 65% 

Same or less interest 35 



35. If you had any choice, which would you prefer to teach? 

Underachievers 20% 

Normally achieving pupils 58 

Superior pupils 22 

36. How much time did you spend disciplining children? 

Less than usual 61% 

Same or more than usual 39 

37. Compared with pupils you taught previously, what was the receptivity 
to learning of those you had this summer? 

More than usual 
V Same as usual 
Less than usual 



36% 

32 

32 




- 28 - 



38. Did you supplement your responses to this questionnaire with 
handwritten comments? 

Yes 17% 

No 83 

One problem in the interpretation of the free comments was 
determining whether they represent the opinions of the majority or of a 
vocal minority. On the basis of the last item» writers and non-writers 
were differentiated and the responses compared. Patterns of response for 
the two groups were almost identical. 
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